Nonlinear dynamics of pattern recognition and optimization

We associate learning in living systems with the shaping of the velocity vector field of a dynamical system in response to external, generally random, stimuli. We consider various approaches to implement a system that is able to adapt the whole vector field, rather than just parts of it - a drawback of the most common current learning systems: artificial neural networks. This leads us to propose the mathematical concept of self-shaping dynamical systems. To begin, there is an empty phase space with no attractors, and thus a zero velocity vector field. Upon receiving the random stimulus, the vector field deforms and eventually becomes smooth and deterministic, despite the random nature of the applied force, while the phase space develops various geometrical objects. We consider the simplest of these - gradient self-shaping systems, whose vector field is the gradient of some energy function, which under certain conditions develops into the multi-dimensional probability density distribution of the input. We explain how self-shaping systems are relevant to artificial neural networks. Firstly, we show that they can potentially perform pattern recognition tasks typically implemented by Hopfield neural networks, but without any supervision and on-line, and without developing spurious minima in the phase space. Secondly, they can reconstruct the probability density distribution of input signals, like probabilistic neural networks, but without the need for new training patterns to have to enter the network as new hardware units. We therefore regard self-shaping systems as a generalisation of the neural network concept, achieved by abandoning the "rigid units - flexible couplings'' paradigm and making the vector field fully flexible and amenable to external force. It is not clear how such systems could be implemented in hardware, and so this new concept presents an engineering challenge. It could also become an alternative paradigm for the modelling of both living and learning systems. Mathematically it is interesting to find how a self shaping system could develop non-trivial objects in the phase space such as periodic orbits or chaotic attractors. We investigate how a delayed vector field could form such objects. We show that this method produces chaos in a class systems which have very simple dynamics in the non-delayed case. We also demonstrate the coexistence of bounded and unbounded solutions dependent on the initial conditions and the value of the delay. Finally, we speculate about how such a method could be used in global optimization

Dynamical system with plastic self-organized velocity field as an alternative conceptual model of a cognitive system

It is well known that architecturally the brain is a neural network, i.e. a collection of many relatively simple units coupled flexibly. However, it has been unclear how the possession of this architecture enables higher-level cognitive functions, which are unique to the brain. Here, we consider the brain from the viewpoint of dynamical systems theory and hypothesize that the unique feature of the brain, the self-organized plasticity of its architecture, could represent the means of enabling the self-organized plasticity of its velocity vector field. We propose that, conceptually, the principle of cognition could amount to the existence of appropriate rules governing self-organization of the velocity field of a dynamical system with an appropriate account of stimuli. To support this hypothesis, we propose a simple non-neuromorphic mathematical model with a plastic self-organized velocity field, which has no prototype in physical world. This system is shown to be capable of basic cognition, which is illustrated numerically and with musical data. Our conceptual model could provide an additional insight into the working principles of the brain. Moreover, hardware implementations of plastic velocity fields self-organizing according to various rules could pave the way to creating artificial intelligence of a novel type

Supplementary Information Files for Delay-induced homoclinic bifurcations in modified gradient bistable systems and their relevance to optimization

Supplementary Information Files for Delay-induced homoclinic bifurcations in modified gradient bistable systems and their relevance to optimizationNonlinear dynamical systems with time delay are abundant in applications but are notoriously difficult to analyze and predict because delay-induced effects strongly depend on the form of the nonlinearities involved and on the exact way the delay enters the system. We consider a special class of nonlinear systems with delay obtained by taking a gradient dynamical system with a two-well “potential” function and replacing the argument of the right-hand side function with its delayed version. This choice of the system is motivated by the relative ease of its graphical interpretation and by its relevance to a recent approach to use delay in finding the global minimum of a multi-well function. Here, the simplest type of such systems is explored for which we hypothesize and verify the possibility to qualitatively predict the delay-induced effects, such as a chain of homoclinic bifurcations one by one eliminating local attractors and enabling the phase trajectory to spontaneously visit vicinities of all local minima. The key phenomenon here is delay-induced reorganization of manifolds, which cease to serve as barriers between the local minima after homoclinic bifurcations. Despite the general scenario being quite universal in two-well potentials, the homoclinic bifurcation comes in various versions depending on the fine features of the potential. Our results are a pre-requisite for understanding general highly nonlinear multistable systems with delay. They also reveal the mechanisms behind the possible role of delay in optimization.<br

Delay-induced homoclinic bifurcations in modified gradient bistable systems and their relevance to optimization

Nonlinear dynamical systems with time delay are abundant in applications but are notoriously difficult to analyze and predict because delay-induced effects strongly depend on the form of the nonlinearities involved and on the exact way the delay enters the system. We consider a special class of nonlinear systems with delay obtained by taking a gradient dynamical system with a two-well “potential” function and replacing the argument of the right-hand side function with its delayed version. This choice of the system is motivated by the relative ease of its graphical interpretation and by its relevance to a recent approach to use delay in finding the global minimum of a multi-well function. Here, the simplest type of such systems is explored for which we hypothesize and verify the possibility to qualitatively predict the delay-induced effects, such as a chain of homoclinic bifurcations one by one eliminating local attractors and enabling the phase trajectory to spontaneously visit vicinities of all local minima. The key phenomenon here is delay-induced reorganization of manifolds, which cease to serve as barriers between the local minima after homoclinic bifurcations. Despite the general scenario being quite universal in two-well potentials, the homoclinic bifurcation comes in various versions depending on the fine features of the potential. Our results are a pre-requisite for understanding general highly nonlinear multistable systems with delay. They also reveal the mechanisms behind the possible role of delay in optimization